[fix](parquet) Fix wrong encoding for parquet page v2#63305
Open
Gabriel39 wants to merge 3 commits into
Open
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
|
run external |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 31130 ms |
Contributor
TPC-DS: Total hot run time: 169421 ms |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary: Add a BE unit test for rejecting Parquet DATA_PAGE_V2 chunks that contain non-dictionary data page encodings in encoding_stats.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- Added BE UT for RowGroupReader::is_dictionary_encoded DATA_PAGE_V2 encoding_stats handling. Attempted ./run-be-ut.sh --run --filter=ParquetThriftReaderTest.is_dictionary_encoded_rejects_plain_data_page_v2, but local environment has JAVA_HOME on JDK 11 and run-be-ut.sh requires JDK 17. Attempted build-support/clang-format.sh, but local environment lacks llvm@16.
- Behavior changed: No
- Does this need documentation: No
Contributor
Author
|
run buildall |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
TPC-H: Total hot run time: 31243 ms |
Contributor
TPC-DS: Total hot run time: 168762 ms |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
1 similar comment
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
/review |
zhangstar333
approved these changes
May 18, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review result: request changes.
Critical checkpoint conclusions:
- Goal/test proof: The production change aims to reject non-dictionary DATA_PAGE_V2 pages for Parquet dictionary filtering, and the main code change is directionally correct, but the added BE unit test does not compile because it calls a private method directly.
- Scope/focus: The production change is small and focused. The test access pattern needs to be corrected without broadening behavior unnecessarily.
- Concurrency/lifecycle/config/compatibility: No new concurrency, special lifecycle, configuration, storage format, or FE-BE protocol compatibility concerns found.
- Parallel code paths: There is a similar Iceberg dictionary-encoding helper/test path that already covers DATA_PAGE_V2; no additional blocking issue found beyond the new Parquet test compile failure.
- Special checks/error handling/memory/transactions/data writes: No new concerns found in the changed production logic.
- Tests/results: Regression coverage was added, but BE unit coverage is currently broken at compile time, so it cannot validate the fix until the private-access issue is resolved.
- Observability/performance: No additional observability or performance issue found for this small metadata check.
User focus: No additional user-provided review focus was present.
| position_delete_ctx, lazy_read_ctx, nullptr, column_ids, | ||
| filter_column_ids); | ||
|
|
||
| EXPECT_FALSE(row_group_reader.is_dictionary_encoded(column_metadata)); |
Contributor
There was a problem hiding this comment.
This test will not compile as written: RowGroupReader::is_dictionary_encoded is declared in the private section of vparquet_group_reader.h, and this test file does not use a friend/test helper or a private visibility override. Because the new BE UT directly calls row_group_reader.is_dictionary_encoded(...), the target fails at compile time instead of validating the DATA_PAGE_V2 fix. Please expose this logic through a testable helper (or test through the public reader path) before adding the assertion.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)